System and method to facilitate creation of educational information

ABSTRACT

The computer-implemented system to facilitate creation of educational information employs a networked computer system that stores at least one resource in association with a first electronic file, storing metadata information about usage of the resource. A resource assembly tool implemented by a computer is programmed to access the networked computer system to display information to an instructor about at least one resource, including the metadata information. This aids the instructor in selecting resources for inclusion in educational information being created. The resource assembly tool is configured to assemble the educational information to include resources selected for inclusion being created for dissemination to learners via said networked computer system. The networked computer system is further configured to capture information about usage of the resource and to update the stored metadata information to reflect said captured information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U. S. patent application Ser. No.10/130,864 filed on Nov. 4, 2002. This application claims the benefitand priority of PCT/US00/24943, filed Sep. 12, 2000. This applicationclaims the benefit of U.S. Provisional Application No. 60/167,887 filedNov. 26, 1999. The entire disclosure of each of the above applicationsare incorporated herein by reference.

GOVERNMENT LICENSE RIGHTS

This invention was made with Government support under 9752985 awarded byThe National Science Foundation. The Government has certain rights inthe invention.

FIELD OF THE INVENTION

This invention relates to a system and method for interactive, adaptive,customized and individualized computer-assisted instruction of students,preferably implemented on network connected computers. Moreparticularly, the system and method comprises i) an assembly tool forbringing diverse educational resources together to create customizedcourse material for the instruction of students and ii) a replicationelement to update each resource and assure access to each updatedresource. The system and method of the present invention responds to theinstructor's creativity, allowing the instructor to shape and controlthe instructional materials and process.

BACKGROUND OF THE INVENTION

The application of computers in education has been limited by severalproblems, including a) a failure to provide systems that adapt to newadvances in course material caused by technological developments, b) afailure to permit customized design of instructional information by theteacher, and c) a failure to integrate systems effectively into theexisting curriculum.

Current approaches merely sequence students through pre-packagededucational materials. These systems do not provide any means forgathering or using more comprehensive information from outside sources.Consequently, the educational materials are relatively static andoutdated, resulting in course of limited to poor quality.

Computer assisted instruction systems have ignored or under-utilizedsuch important developments in computer technology in recent years asclient-server systems and networking systems. Though now an active fieldwith a wide spectrum of activities from research to commercialapplications, application of dynamic on-line systems in educational,instructional, and homework tasks is only just starting to be explored.

What is needed is a computer-driven system that adapts to newinformation, allows for new information to be readily utilized by aninstructor, so that the instructor can integrate new information intothe existing curriculum.

SUMMARY OF THE INVENTION

The present invention contemplates a system and method forcomputer-assisted education, comprising a Resource Assembly Tool (RAT)and a Dynamic Resource Replication (DRR) element. The RAT provides agraphical user interface inside of a standard web browser. The interfaceconsists of a plurality (e.g. two or more) browser windows. In oneembodiment, a first browser window is configured as resizable with aframeset that contains the menu and the map under construction, and asecond multipurpose non-resizable window that displays information andinput forms. Once a new map is started, the author can then enlarge themap area and insert resources into it. These resources may be identifiedby URL or simple network browsing or searching metadata sources. Onceresources have been located they are connected together using the linkmode of the RAT.

Rather complex maps can be generated using the RAT. These are differentfrom binary trees, both because branches can loop back, and becausebranches can be re-united. Additionally, maps that are not created bythe RAT are accessible by the RAT with the resultant generation of agraphical layout of the map. The RAT then provides the integration ofseveral maps, thus creating nested maps. A course can easily containseveral hundred of these nested maps. Due to the inherent complexity ofthis arrangement all maps and conditions are compiled into apre-processed binary data structure at the start of a session.

The delivery system for these resources consists of a distributednetwork of servers. While each author has a so-called home server whichholds the authoritative copy of all resources published by that author,all servers in the network can host sessions utilizing these resources.DRR is designed to prevent overload situations on any one server in thenetwork, as well as to avoid single points of failure in the network.Replication processes between servers in the network are triggered intwo situations: i) a user wants to retrieve a resource from a server inthe network which is not locally present on that server; ii) an authorpublishes a new version of a resource. In the first case, afterlocalizing the author's home server and a series of authenticationsteps, the session-hosting server subscribes to the resource andreplicates it to a local copy. In the latter case, all subscribedservers are notified that a new version is present, and according to adecision algorithm either replicate the new version or unsubscribe fromit.

As the RAT allows an author to assemble resources from across thenetwork into consistent presentations, the DRR mechanism was found to becapable of ensuring the integrity of such presentations even if parts ofthe network are down, inaccessible or overloaded. The dynamic nature ofthe DRR allows for just-in-time just-enough replication of resources andthus is different from caches, which are not updated when a resourcechanges, and static replication mechanisms implemented in many databaseswhich only allow for non-discriminate replication of predefined chunksof data at prespecified times.

The characteristics of the DRR allow extremely large server networks tooperate together without risk of a network failure. The selectivereplication capability of the DRR of the specific file requested by RATlimits the required memory, transmission time, and does not require theusers computer to interact with the complete database of the Libraryserver.

Thus, the present invention contemplates, in one embodiment, a method ofcombining educational resources, comprising: a) providing i) a networkcomprising a first home server of a first educational institution and asecond home server of a second educational institution, said first homeserver hosting (i.e. holding in memory or in otherwise in a database) afirst pool of resources, said second home server hosting to a secondpool of resources, said resources comprising combinable fragments,pages, sequences and courses, ii) a first author at said firsteducational institution and a second author at said second educationalinstitution, iii) a resource assembly tool configured for use by saidfirst and second authors; and b) accessing a resource (e.g. a fragment,page, sequence, portion of a course, or entire course) from said secondpool of resources through the actions of said first author; c)combining, through the use of said resource assembly tool by said firstauthor, said resource from said second pool of resources with a resource(again, a fragment, page, sequence of pages, portion of a course, orentire course) from said first pool of resources. In a preferredembodiment, said accessing comprises replicating said resource from saidsecond pool of resources.

It is not intended that the present invention be limited to thecombination of resources “as is.” The author has the freedom to makechanges to the fragments, pages, sequences, courses and the like havingeducational content. For example, prior to said combining, the presentinvention contemplates that said first author modifies said resource ofsaid second pool of resources (the original resource is not overwritten;rather a new version is created). On the other hand, prior to saidcombining, said first author may modify said resource of said first poolof resources. Of course, said first author may also choose to modifyboth resources before combining.

The present invention contemplates that, in one embodiment, saidcombining is part of the process for creating a first page, said pagecomprising combined resource fragments. It is not intended, however,that the present invention be limited to pages having only twofragments. In other words, the combining of first and second fragmentsmay be preceeded—or followed—by combinations of other fragments. In somecases, the combining is the last step in the creation of a first authorpage.

It is also not intended that the combining only be at the fragmentlevel. Resources can be combined to create entire courses. For example,a sequence of pages can be combined with another sequence to make acourse.

As noted in more detail below, the present invention contemplates thatsaid first author publishes said first author combination (e.g. page,sequence of pages, or course), thereby adding said first authorcombination to said first and second pools of resources. In other words,the product of the combining is made accessible to authors from otherinstitutions, so that they may use the new product “as is” or assubsequently modified. For example, the first author page may beaccessed by said second author and said second author, after saidaccessing, is free to modify said first author page so as to create asecond author page.

It is not intended that the second author be limited to the nature orextent of modifications. The second author may choose to delete or editmaterial on the page. On the other hand, the second author may choose toadd additional fragments, pages, etc. from one or more pools ofresources. In any event, the present invention contemplates that saidsecond author, after said accessing and modifying, publishes said secondauthor combination (e.g. page or other resource), thereby adding saidsecond author combination to said first and second pool of resources.

In other words, the sharing can continue because—whatever the secondauthor does to the material—it is available to the first author forpossible use, as well as any other authors that are part of the systemvia the network. Indeed, it is not intended that the number ofinstitutions or authors be limited in any way. Instances where three ormore institutions are involved are contemplated. For example, thepresent invention contemplates in one embodiment a method of combiningeducational resources, comprising: a) providing i) a network comprisinga first home server of a first educational institution, a second homeserver of a second educational institution, and a third home server of athird educational institution, said first home server hosting a firstpool of resources, said second home server hosting a second pool ofresources, and said third home server hosting a third pool of resources,said resources comprising combinable fragments, pages, sequences ofpages, and courses, ii) a first author at said first educationalinstitution and a second author at said second educational institution,iii) a resource assembly tool configured for use by said first andsecond authors; and b) accessing, through the actions of said firstauthor, i) a resource (e.g. fragment, page, etc.) from said second poolof resources and ii) a resource from said third pool of resources; c)combining, through the use of said resource assembly tool by said firstauthor, said resource from said second pool of resources and saidresource from said third pool of resources to create a firstcombination.

In the above-described embodiment, said first author is using andcombining material from third-party sources. However, it is not intendedthat the present invention limit the author as to where resources can beobtained. Moreover, the author can combine in any manner desired. Forexample, the method can further comprise d) combining a resource (e.g. afragment, page, etc.) from said first pool of resources with said firstcombination to create a second combination.

All of the above-described discussion about replicating as part ofaccessing can apply to the three (or more) institution embodiment.Similarly, the freedom to modify prior to combining should again beunderscored. Most importantly, the present invention contemplates thatsaid first author publishes said first author page, thereby adding saidfirst author page to said first, second and third pools of resources,and thereby permitting access by other authors (who may make furtherchanges and combinations to create a second author page).

A variety of system configurations can carry out the methods describedabove. In one embodiment, the present invention contemplates a systemfor sharing educational resources, comprising: a) a first authorcomputer at a first educational institution connected to a first homeserver, said first author computer having a user interface, said firsthome server providing access to a first pool of resources comprisingcombinable resources (e.g. fragments, pages, sequences of pages,portions of courses, and entire courses); b) a second author computer ata second educational institution connected to a second home server, saidsecond author computer having a user interface, said second home serverproviding access to a second pool of resources comprising combinableresources; c) a network connecting said first home server of said firsteducational institution to said second home server of said secondeducational institution; d) a resource assembly tool configured for usethrough said user interface of said first and second author computers,said resource assembly tool capable of combining said combinableresources from said first pool of resources and said second pool ofresources; and e) a resource replication element configured so as toreplicate resources from said first and second pools of resources priorto said combining of said resource assembly tool. Importantly, thesystems are not limited to the number of servers or client computers.

DESCRIPTIONS OF THE DRAWINGS

FIG. 1 provides a top level schematic of the primary interfaces of theLearningOnline Network.

FIG. 2A shows the overview of network commuication links between serversof the LearningOnline Network.

FIG. 2B shows an example of the Hosts Lookup Table.

FIG. 2C illustrates the response times of server-server communicationswithout disk access.

FIG. 2D illustrates the response times of server-server communicationswith disk access.

FIG. 3A depicts the implementation of the Dynamic Resource Replication.

FIG. 3B describes the process of modifying a resource.

FIG. 4 shows a top level illustration of the Resource Assembly Tool.

FIG. 5A presents an embodiment of a graphical user interface duringResource Assembly Tool access.

FIG. 5B presents an example of the graphical user interface duringResource Assembly Tool access.

FIG. 5C presents another example of the graphical user interface duringResource Assembly Tool access.

FIG. 6 shows the non-graphical (XML) presentation of FIG. 5C.

FIG. 7A shows an example of a course map having nested sequences.

FIG. 7B shows an example of a course map sequence having nested pages.

FIG. 7C shows an example of a course map summary sequence.

FIG. 8 shows an example of a flow chart during a course initializationfirst run.

FIG. 9A illustrates an example of a flow chart during a courseinitialization first run for the procedure loadmap.

FIG. 9B demonstrates an example of a resource properties hash dumpresulting from the loadmap procedure.

FIG. 9C demonstrates an example of a resource properties hash dumpresulting from the links between resources from the loadmap procedure.

FIG. 9D demonstrates an example of a resource properties hash dumpresulting from the links and link conditions between resources from theloadmap procedure.

FIG. 10 depicts an example of an excerpt of the dump of the conditionarray constructed in the loadmap procedure.

FIG. 11A illustrates an example of a flow chart during a courseinitialization first run for the procedure traceroute.

FIG. 11B depicts an example of an excerpt of the resource propertieshash dump resulting from cumulative link conditions to reach a certainresource.

GENERAL DESCRIPTION OF THE INVENTION

Current computer-assisted instructional systems have only haphazardlyexploited the potential of client-server systems and networkingtechnologies. The present inventors recognize that systems, runningunder sophisticated windowing operating systems, can support advancedobject based software applications, including high speed graphics,animation and audio output, that are particularly suited to education.Servers can store gigabytes of educationally relevant data and programsat central or distributed locations at quite reasonable cost.

Clients and servers can be linked remotely with increasing convenienceand decreasing cost. The Internet has emerged as a means of providing aninexpensive means of connecting computers to provide effectivecommunications and access to information and other resources such assoftware. Further Internet developments that made the Internet trulyuniversal include the HTML and the HTTP protocols, which provideplatform independent access to hyperlinked multimedia information, andthe Java™ programming language, which provides platform independentsoftware for Internet applications programming. Subsets of the Internet(called intranets) have become an increasingly important means fordisseminating information and enabling communication within constraineddomains, such as a single school system or corporate enterprise.

The present invention provides the tools for management and control overthe computer-assisted instruction materials and provides the neededflexibility to allow an instructor to construct customized materialusing information from diverse educational resources. The presentinvention provides the tools for the integration of traditionaleducational material (such as data, equations and the like) from severalsources. More importantly, the present invention permits an instructorto select from a wider and richer variety of educationally relevantsound and visual display objects. All elements of the on-screen displaycan be pulled from diverse sources and synthesized in an integrateddisplay calling for graphics, animation, video, or sound as appropriate.The present invention provides the authoring tools needed to generatemultimedia educational course material presentations that is accessibleto the on-line student.

The elements of the display objects can be created by people other thanthe actual instructor (e.g. third-party teachers, artists, animators,singers and so forth). The instructor, through the tools of the presentinvention, has access to these materials and can utilize all or aportion of a third-party's course materials in the assembly of acustomized on-line course, lecture, class, or session. The educationalresources are stored in libraries (e.g. as data snips or dynamic clipart) and then accessed by the instructor in an implementation of thisinvention. These educational resources can be in the form of short clipsof text, sound, voice, graphics, animation or video. Using the resourceassembly tool of the present invention, these diverse resources can becombined according to the desires and creativity of the instructor togenerate a customized presentation.

Another important object is that the method and system of this inventionis adapted to implementation on a variety of networks. When soimplemented, the interactive, adaptive, and self-paced computer-assistedinstruction and homework provided by this invention is available togeographically dispersed students and from geographically dispersedschools.

The network on which this invention is implemented can be configured asan intranet. In another embodiment, implementation is achieved over thepublic Internet. In either case, the system is configured withappropriate links and is compatible with browser and e-mail formatextensions. In short, this invention is adaptable to Network Computers(“NC”). NCs are low cost computers specifically designed to accessintranets or the public Internet. In a current preferred embodiment andimplementation, this invention is adaptable to multimedia PCs for somestudents or students with special needs. Typical interactive devicesinclude keyboards, mice or other pointing devices, voice recognition,joy-sticks, touch activated devices, light-pens, and so forth. Otherdevices, such as virtual reality devices, can be added as they becomecommercialized.

Authoring instructional materials for a course, lecture, class or othertype of educational presentation to a student, when done according tothe method of the present invention, typically comprises several steps,including decisions about the objects to display to the student and thesequencing of these objects. The first step is the selection of objectswhich carry the education content for presentation to a student. Objectscan include visual display items, such as text, animation or movies,audible display items, such as voice, audio and so forth. They caninclude input items known in the computer arts, such as buttons toselect. Selections to chose from include (but are not limited to) thetext to enter, the useful hypertext and hypermedia links, and functionsto perform with student input and so forth. The second step is theselection of the sequencing logic for the ordered display of the objectsto the student throughout the course, lecture, class or session.

Importantly, to increase the utility of the materials, the number ofhard-coded hyperlinks between the resources should be minimized. Theactual combining and sequencing is part of the system functionality anddriven by RAT-constructed ‘roadmaps’, which are constructed by theinstructors. With this mechanism, one and the same resource can be partof different courses in different contexts.

The present invention contemplates the use of algorithms in the designof the student interface virtual course resources. In one embodiment,the learner is provided with multiple representations of the sameknowledge elements and can select a preferred representation. In apreferred embodiment, algorithms that learn from a learner's previousselection of preferred options are employed and these automaticallycustomize the course to the learners' needs, offering remedial actionsfor detected shortcomings and allowing leaps over segments of materialfor which the student is predicted to already have achieved mastery.

The present invention contemplates an automated exam engine that willproduce randomized and/or individualized tests without the instructorhaving the need or even the opportunity to select the problems. This canbe done by providing a large pool of exam questions via an open sourcedatabase. Each exam problem contains attached metadata that catalogs itsdegree of difficulty and discrimination for students of differentability levels.

Using the RAT, an instructor can create and/or assemble a customized setof assignments, quizzes, and examinations with a large variety ofconceptual questions and quantitative problems. These on-linepresentations can include pictures, animations, graphics, tables, links,etc. The writing and development is facilitated by numerous types ofindividualized problems designed to encourage students to collaborateand discuss concepts while insuring that problems differ for eachstudent to inhibit rote copying.

Indeed, the present invention contemplates a feedback system i) wherestudents and other instructors can comment, criticize, evaluate and/orgrade a resource, and ii) where the author of the resource can comment,evaluate, grade and/or assist with performance of the student. Withregard to the latter, in a preferred embodiment, students are giveninstant feedback and relevant hints via the Internet and may correcterrors without penalty prior to an assignment's due date. The systemkeeps track of student's participation and performance, and records areavailable in real time both to the instructor and to the individualstudent. Statistical tools and graphical displays facilitate assessmentand course administration.

This invention contemplates the ability of this student feedback systemto be initiated by a screen button on the student's navigation graphicalinterface that opens up a text field. When the student sends a feedback,it arrives at an email address (or set of email addresses) specified bythe course faculty; independent of the computer platform. With thenormal ‘reply’ function, faculty or teaching assistants can respond tothe student input and the reply automatically is returned by handlingwithin the network. This type of student input improves the use of theon-line resources. For example, if additional specific hints areincluded in the reply by the instructor, these changes take effectwithin the network immediately and all students benefit.

Also contemplated by this invention is a chatroom that allows formultiple ways of communication. Learners can post text, graphics,whiteboard information and formulas into the chatroom, which operateswithout any plugins.

With regard to students and other instructors commenting and gradingresources, the present invention contemplates compiling feedback of thistype in an electronic file associated with the particular resource(whether the resource is a fragment, page, sequence of pages, portion ofa course, or entire course). When the author of the resource (or anotherinstructor) accesses the resource, the cumulative feedback associatedwith the particular resource (at that time) is available for review(e.g. the author or other instructor can query the feedback) so thatcomments and criticism can be considered in any effort to edit/modifythe resource, thereby improving the resource. This process is anon-going process, allowing for the possibility of continuous improvementof the resource as it is experience by students or utilized by otherinstructors. By virtue of the feedback system, resources can be placedin competition. That is to say, students can apply a grade (or a set ofgrades directed to various features of a resource such as clarity,technical functionality, accuracy and the like) to a particular resource(e.g. a fragment, page, sequence of pages, portion of a course, orcourse) and that grade can be compared to the grade given to anotherresource in the same field (e.g. biology, math, etc.). If desired, theresource receiving the higher grade (or the higher average grade basedon cumulative grading from a plurality of students) can thereafter beselected preferrentially for instructional purposes.

Thus, the present invention also contemplates a method of evaluatingeducational resources, comprising: a) providing i) a network comprisinga first home server of a first educational institution and a second homeserver of a second educational institution, said first home serverhosting a first pool of resources, said second home server hosting asecond pool of resources, said resources comprising combinablefragments, pages, sequences and courses, ii) a first author at saidfirst educational institution and a second author at said secondeducational institution, iii) a resource assembly tool configured foruse by said first and second authors; iv) one or more students connectedthrough one or more computers to said network, said computers having auser interface; b) displaying a resource from said first pool ofresources through the actions of a student on said user interface ofsaid student's computer, to create a first displayed resource; c)grading said first displayed resource, whereby a numerical valueassociated with said first displayed resource is stored in a file; andd) accessing said first displayed resource (whether or not the resourceis currently displayed or otherwise in use) through the actions of saidfirst author, under conditions such that said numerical value associatedwith said first displayed resource is apparent to said author. Ofcourse, the process need not stop here. In one embodiment, the presentinvention further contemplates, e) combining, through the use of saidresource assembly tool by said first author, a resource from said firstor second pool of resources with said displayed resource underconditions wherein said displayed resource is modified to create amodified first displayed resource; f) displaying said modified firstdisplayed resource through the actions of a student on said userinterface to create a second displayed resource; g) grading said seconddisplayed resource, whereby a numerical value associated with saidsecond displayed resource is stored in a file; and h) accessing saidsecond displayed resource (whether or not the resource is currentlydisplayed or otherwise in use) through the actions of said first author,under conditions such that said numerical value associated with saidsecond displayed resource is apparent to said author. In some cases, thenumerical value associated with the second displayed resource will behigher than the numerical value associated with the first displayedresource, indicating that [at least according to the student(s)] theresource has been improved. Again, the process need not stop here;indeed, the above indicated steps can be repeated numerous times (in themanner of a feedback loop).

The present invention is not limited to grading from students. Theabove-indicated method steps can be modified such that other instructorsprovide the feedback and the grading.

In addition, the present invention is not limited to grading andnumerical values. The above-indicated method steps can be modified suchthat written comments or other symbols are provided as feedback inassociation with a resource.

Furthermore, the present invention is not limited to review of feedbackand use of feedback by the original author. The above-indicated methodsteps can be modified such thait a second author at another institutionviews the feedback and modifies the resource accordingly.

A. The Resource Assembly Tool

The RAT generates pathways that link resources. The RAT acts as agraphical user interface inside of a standard web browser. Thisinvention contemplates that any browser-compatible software languagewill support RAT's function. The RAT/browsers interface in oneembodiment comprises two or more windows. For example, one window isresizable, and the other is a multipurpose non-resizable window. Theformer window contains the menu and the current project, while thelatter window displays general operational information. When a newproject is started the default settings are limited to start and finishcodes, thus allowing the user complete freedom of choice for theresource links.

The project window operates in a similar manner as most popular computeroperating systems. Editing is accomplished by using the mouse to clickon the appropriate resource, or in the alternative, a dialog box allowsuse of describing the title of the resource for retrieval through thenetwork. The dialog boxes also accept URL addresses that are either partof the institutions intranet or an external WWW URL. The inventioncontemplates an ability for the user to access a central databasedirectory to search for and/or browse to locate desired resources.

The dialog box may also be utilized to delete resources from a linkedpathway as well as adding them. It is envisioned that an option of acomplete severance from the integrated pathway is coupled with analternative options of merely removing the specific resource and leavingthe pathway intact.

Once a desired resource is located, the link mode of the RAT isutilized. This function “physically” connects on resource to anotherthat is the basis of the basic inventive concept of this system.

The default mode of the RAT is info mode. This allows the user toquickly pan the mouse over the presented pathway to examine the metadataof each resource. For example, if the mouse pointer is stopped over amovie resource, an information bubble will appear that presents thetitle, actors, subject matter, and running time. This will allow aninstructor to quickly assess whether any particular pathway requiresmodification for a new course or due to changes within a specificacademic field. If the instructor does require changes to the resourcethey only click on the resource and the RAT enters edit mode, thusallowing changes.

B. The Replication Element

To enable immediate and dynamic system reconfiguration in case of serveror network downtimes and overload situations, data replication isrequired, where any machine in the network can serve any learner in theinstitution. On approach is to use a server network of inexpensive webservers running Linux™ and the Apache web server, which communicate witheach other via persistent TCP connections. The network of the presentinvention has the ability to replicate resources and update user recordsdynamically from server to server, as well as the ability to transferuser sessions between each other in overload situations.

In order for the resource pool to be functionally more consistent andcomprehensive, all resources in one embodiment take the form of amultimedia object and are stored in the multimedia resource pool. Foreach content author, the system will provide separate privateconstruction and public resource space. Moving a resource fromconstruction space into the public resource pool is combined with the‘wizard’-assisted gathering of abstracts, classification information andkeywords (IMS compliant metadata and the Library of Congressclassification scheme), as well as versioning and access privilegescheme.

C. Other System Elements

This invention contemplates the integration of the Computer-AssistedPersonalized Approach (CAPA) with the other system elements in order toprovide students with personalized problem sets, quizzes and exams.Different students see slightly different problems, which enables themto collaborate on a conceptual level without being able to blindly copyanswers. Students are given instant feedback and relevant hints via theInternet and may correct errors without penalty until the assignment duedate. The system records the students' participation and performance,and the records are available online to both the instructor and theindividual student.

CAPA is a teaching tool, not a curriculum, and as such does not dictatecourse design, content or goals. Instead, it enables faculty to augmenttheir courses with individualized relevant exercises. CAPA, as astand-alone system has been widely accepted by more than 40,000 studentsin astronomy, biochemistry, chemistry, mathematics, physics, botany,accelerator physics, and a host of human ecology and computer sciencecourses since 1992. CAPA has been licensed by various institutions forinstruction in several disciplines.

This invention contemplates an integrated embodiment of CAPA with theLearningOnline Network (LON-CAPA). This linkage implements aninfrastructure that allows a group of organizations (departments,universities, colleges, and commercial businesses) to link their on-linelearning communities. LON-CAPA thus enables institutions to share theiron-line learning objects and act as a common marketplace.

Most current on-line homework engines are close to faculty needs anddesires but due to time and budget constraints they lack scalability andfailover security. Systems that provide on-line homework are frequentlysubject to strong peak workloads close to deadlines, while at the sametime their functionality is crucial.

The LON-CAPA is a distributed system with a classical three-tierarchitecture based on a communication backbone. The nodes in the networkcan be geographically distributed among different departments and eveninstitutions with only the commodity Internet as link.

LON-CAPA is based on a network of basic computers as access servers anda few high-performance library servers. While library servers hold allobjects (content and system) of a subset of users within the networkanytime, access servers act as intelligent caches and replicate theneeded resources at that point upon demand and update them as becomesnecessary.

Definitions

The terms “instructional materials” or “educational materials” encompassall educational resources used as components of a course of instruction,or as components of a lecture, class, or other type of session with oneor more students.

The terms “educational resource” or simply “resource” indicate anelementary piece or “fragment” of text, sound, voice, graphics,animation, and/or video (e.g. in the form of data snips or clip art)that can be combined, utilizing the Resource Assembly Tool (RAT) of thepresent invention, to represent a complete on-screen presentation (e.g.whether a single screen or “page of fragments” or whether a plurality ofpages and links). With RAT, fragments can be combined into “pages” (i.e.something that a browser would display as a page or a printer wouldprint as a page). Pages can be combined into sequences, and sequencesinto courses. Such components are selected according to the course(physics, math, biology, etc.) and can include prerequisite tests,pretests, lessons, and unit tests.

A “network” is the hardware and software connecting student clientcomputers to school servers. “Network connections” can comprise fiberoptic links, wired links or terrestrial or satellite wireless links.Servers are linked together on the network. There are “home” servers foreach resource “author.” A first resource author can, using the ResourceAssembly Tool of the present invention, access a resource on his/her ownhome server; alternatively, a first resource author can also access aresource on a second resource author's home server.

Teachers and other instructional designers can create, or “author,”materials for use in this invention (teachers are thus “authors”).Materials can be original or can be derived from existing textbooks,workbooks or media material. They can simply employ elements ofstandardized curricula, pretests such as criterion tests, post-tests,and standardized texts. However, the present invention is particularlysuited to non-standardized curricula and the use of on-line educationalresources authored by third-parties collaborating in a combinededucational effort.

Parties collaborating in a combined educational effort can be from avariety of “educational institutions” including but not limited topublic and private colleges and universities.

To encourage resource sharing and “re-usage” and to improve the qualityof the educational resources, teachers and other instructional designersshould be able to modify (“edit”) the resources for their own use, oreven be ‘value adders’. Modifying and adding value to a resource allowsall parties collaborating and contributing to the resource library toshare and generate improved and enhanced instructional materials. Itshould be stressed that “modifying” is meant to indicate the creation ofa new derivative resource of branch (while preserving the originalresource in its original form). In other words, the modified resourcedoes not override the a resource; rather it creates a new resource inthe system which is derived from the original resource. In a preferredembodiment, a detailed log is kept with all branches of a resourcespecifying authorship history in machine-readable form.

Efficient re-usage of educational objects only works if those objectscan actually be found in the potentially large pool of resources. Thepresent invention contemplates that an instructor can “query” for animage with a graphical representation of particular information form aparticular topic (e.g. trigonometry, calculus, etc.).

The present invention contemplates “personalized homework” or“individualized homework.” This means that each student sees a slightlydifferent computer-generated problem. This encourages collaborationbetween students on a conceptual level, but prevent blind copying ofanswers. The students get immediate automatic feedback for their enteredhomework answers, while faculty are able to provide answer-specifichints for common problems identified either beforehand or during theclass term.

The term “httpd” is used to indicate Hypertext Transfer Protocol Daemon,a detached permanent server process that serves web content.

The term “GIF” refers to a Graphical Interface Format, a common formatfor INternet graphics developed by Compuserve.

DETAILED DESCRIPTION OF THE INVENTION

In one embodiment of the system and method of the present invention, theNetwork comprises of relatively inexpensive upper-PC-class servermachines which are linked through the commodity interne in aload-balancing, dynamically content-replicating and failover-secure way.FIG. 1 schematically shows an overview of this network. All machines inthe Network are connected with each other through two-way persistentTCP/IP connections. Clients (B, F, G and H in FIG. 1) connect to theservers via standard HTTP. There are two classes of servers, LibraryServers (A and E in FIG. 1) and Access Servers (C, D, I and J in FIG.1). Library Servers are used to store all personal records of a set ofusers, and are responsible for their initial authentication when asession is opened on any server in the Network. For Authors. LibraryServers also host their construction area and the authoritative copy ofthe current and previous versions of every resource that was publishedby that author. Library servers can be used as backups to host sessionswhen all access servers in the Network are overloaded. Otherwise, forlearners, access servers, are used to host the sessions. Library serversneed to be strong on I/O, while access servers can generally be cheaperhardware. The network is designed so that the number of concurrentsessions can be increased over a wide range by simply adding additionalAccess Servers before having to add additional Library Servers.Preliminary tests showed that a Library Server could handle up to 10Access Servers fully parallel.

The Network is divided into so-called domains, which are logicalboundaries between participating institutions. These domains can be usedto limit the flow of personal user information across the network, setaccess privileges and enforce royalty schemes.

Example of Transactions

FIG. 1 also depicts examples for several kinds of transactions conductedacross the Network. An instructor at client B modifies and publishes aresource on her Home Server A. Server A has a record of all servermachines currently subscribed to this resource, and replicates it toservers D and I. However, server D is currently offline, so the updatenotification gets buffered on A until D comes online again. Servers Cand J are currently not subscribed to this resource. Learners F and Ghave open sessions on server I, and the new resource is immediatelyavailable to them. Learner H tries to connect to server I for a newsession, however, the machine is not reachable, so he connects toanother Access Server J instead. This server currently does not have allnecessary resources locally present to host learner H, but subscribes tothem and replicates them as they are accessed by H.

Learner H solves a problem on server J. Library Server E is H's HomeServer, so this information gets forwarded to E, where the records of Hare updated.

Transaction Mechanism

FIG. 2 elaborates on the details of this network infrastructure. FIG. 2Adepicts three servers (A, B and C) and a client who has a session onserver C. As C accesses different resources in the system, differenthandlers, which are incorporated as modules into the child processes ofthe web server software, process these requests.

Our current implementation uses mod_per1 inside of the Apache web serversoftware. As an example, server C currently has four active web serversoftware child processes. The chain of handlers dealing with a certainresource is determined by both the server content resource area (seebelow) and the MIME type, which in turn is determined by the URLextension. For most URL structures, both an authentication handler and acontent handler are registered.

Handlers use a common library lonnet to interact with both locallypresent temporary session data and data across the server network. Forexample, lonnet provides routines for finding the home server of a user,finding the server with the lowest load average (loadavg), sendingsimple command-reply sequences, and sending critical messages such as ahomework completion, etc. For a non-critical message, the routines replywith a simple “connection lost” if the message could not be delivered.For critical messages, lonnet tries to reestablish connections andre-send the command. If no valid reply could be received, it answers“connection deferred” and stores the message in buffer space to be sentat a later point in time. Also, failed critical messages are logged.

The interface between lonnet and the Network is established by amultiplexed UNIX domain socket (denoted DS in FIG. 2A). The rationalebehind this rather involved architecture is that httpd processes (Apachechildren) dynamically come and go on the timescale of minutes, based onworkload and number of processed requests. Over the lifetime of an httpdchild, however, it has to establish several hundred connections toseveral different servers in the Network.

On the other hand, establishing a TCP/IP connection is resourceconsuming for both ends of the line, and to optimize this connectivitybetween different servers, connections in the Network are designed to bepersistent on the timescale of months, until either end is rebooted.This mechanism will be elaborated on below.

Establishing a connection to a UNIX domain socket is far less resourceconsuming than the establishing of a TCP/IP connection. lonc is a proxydaemon that forks off a child for every server in the Network. Whichservers are members of the Network is determined by a lookup table, ofwhich FIG. 2B is an example. In order, these entries denote: an internalname for the server, the domain of the server, the type of the server,the host name, and the IP address.

The lonc parent process maintains the population and listens for signalsto restart or shutdown, as well as USR1. Every child establishes amultiplexed UNIX domain socket for its server and opens a TCP/IPconnection to the lond daemon (discussed below) on the remote machine,which it keeps alive. If the connection is interrupted, the child dies,whereupon the parent makes several attempts to fork another child forthat server.

When starting a new child (a new connection), first an init-sequence iscarried out, which includes receiving the information from the remotelond which is needed to establish the 128-bit encryption key; the key isdifferent for every connection. Next, any buffered (i.e., delayed)messages for the server are sent.

In normal operation, the child listens to the UNIX socket, forwardsrequests to the TCP connection, gets the reply from lond, and sends itback to the UNIX socket. Also, lonc takes care of the encryption anddecryption of messages. lonc was built by putting a non-forkingmultiplexed UNIX domain socket server into a framework that forks aTCP/IP client for every remote lond.

lond is the remote end of the TCP/IP connection and acts as a remotecommand processor. It receives commands, executes them, and sendsreplies. In normal operation, a lonc child is constantly connected to adedicated lond child on the remote server, and the same is true viceversa (two persistent connections per server combination).

lond listens to a TCP/IP port (denoted P in FIG. 2A) and forks offenough child processes to have one for each other server in the networkplus two spare children. The parent process maintains the population andlistens for signals to restart or shutdown. Client servers areauthenticated by IP.

When a new client server comes on-line, lond sends a signal USR1 tolonc, whereupon lonc tries again to reestablish all lost connections,even if it had given up on them before a new client connecting couldmean that that machine came on-line again after an interruption.

The gray boxes in FIG. 2A denote the entities involved in an exampletransaction of the Network. The Client is logged into server C, whileserver B is her Home Server. Server C can be an Access Server or aLibrary Server, while server B is a Library Server. Client submits asolution to a homework problem, which is processed by the appropriatehandler for the MIME type “problem”. Through lonnet, the handler writesinformation about this transaction to the local session data. To make apermanent log entry, lonnet establishes a connection to the UNIX domainsocket for server B. lonc receives this command, encrypts it, and sendsit through the persistent TCP/IP connection to the TCP/IP port of theremote lond. lond decrypts the command, executes it by writing to thepermanent user data files of the client, and sends back a replyregarding the success of the operation. If the operation wasunsuccessful, or the connection would have broken down, lonc would writethe command into a FIFO buffer stack to be sent again later. lonc nowsends a reply regarding the overall success of the operation to lonnetvia the UNIX domain port, which is eventually received back by thehandler.

Scalability and Performance Analysis

The scalability was tested in a test bed of servers between differentphysical network segments and FIG. 2B shows the network configuration ofthis test. In the first test, the simple ping command was used. Thepinging command is used to test connections and yields the server shortname as reply. In this scenario, lonc was expected to be thespeed-determining step, since lond at the remote end does not need anydisk access to reply. The graph in FIG. 2C shows the number of secondsuntil completion versus the number of processes issuing 10,000 pingcommands each against one Library Server (a 450 MHz Pentium II was usedin this test, with a single IDE HD). For the solid dots, the processeswere concurrently started on the same Access Server and the time wasmeasured until the processes finished—all processes finished at the sametime. One Access Server, the 233 MHz Pentium II, can process about 150pings per second, and as expected, the total time grows linearly withthe number of pings.

The gray dots were taken with up to seven processes concurrently runningon different machines and pinging the same server—the processes ranfully concurrent, and each process finished as if the other ones werenot present (about 1000 pings per second). Execution was fully parallel.

In a second test, lond was the speed-determining step—10,000 putcommands each were issued first from up to seven concurrent processes onthe same machine, and then from up to seven processes on differentmachines. The put command requires data to be written to the permanentrecord of the user on the remote server.

In particular, one “put” request meant that the process on the AccessServer would connect to the UNIX domain socket dedicated to the libraryserver, lonc would take the data from there, shuffle it through thepersistent TCP connection. lond on the remote library server would takethe data, write to disk (both to a dbm-file and to a flat-texttransaction history file), answer “ok”, lonc would take that reply andsend it to the domain socket, the process would read it from there andclose the domain-socket connection.

The graph in FIG. 2D shows the results of the above test. Series 1(solid black diamond) is the result of concurrent processes on the sameserver—all of these are handled by the same server-dedicated lond-child,which lets the total amount of time grow linearly. Series 2 through 8were obtained from running the processes on different Access Serversagainst one Library Server, each series goes with one server. In thisexperiment, the processes did not finish at the same time, which mostlikely is due to disk-caching on the Library Server—lond-children whosedatafile was (partly) in disk cache finished earlier. With sevenprocesses from seven different servers, the operation took 255 secondstill the last process was finished for 70,000 put commands (270 persecond)—versus 530 seconds if the processes ran on the same server (130per second).

Server Content Resource Areas

Internally, all resources are identified primarily by their URL.Different logical areas of the server are distinguished by the beginningpart of the URL:

-   /adm: publicly available content, logos, manual pages, etc.-   /res/domainname/authorname/ . . . : the resource area, holding    course maps, HTML pages, homework, movies, applets, etc. Access to    these files is restricted by the cookie-based authentication    mechanism. Content in this area will be served by type-dependent    handlers, for example, one handlers to serve homework problems, and    another one for TeX resources. The structure of this area of the    server is exactly the same on every server, even though not all    resources might be present everywhere.-   /raw/domainname/authorname/ . . . : internally, this is just a    symbolic link to the res directory, however, no content handlers are    called when serving a resource and access is controlled by IP rather    than cookies. This structure is used for replication of resources    between servers.-   /˜authorname/ . . . : the content construction space. This is normal    UNIX filespace, which however can only by viewed on the web by the    authors themselves through the cookie-based authentication. Content    handlers are active for this space. This space can be mounted on    other UNIX machines, as well as AppleShare and Windows. Below the    authorname, this directory has the same structure as the resource    space of the author.    Publication of a Resource

Authors can only write-access the /˜authorname/ space. They can copyresources into the resource area through the publication step, and movethem back through a recover step. Authors do not have directwrite-access to their resource space. During the publication step,several events will be triggered. Metadata is gathered, where a wizardmanages default entries on a hierarchical per-directory base. The wizardimports the metadata (including access privileges and royaltyinformation) from the most recent published resource in the currentdirectory, and if that is not available, from the next directory above,etc. The Network keeps all previous versions of a resource and makesthem available by an explicit version number, which is inserted betweenthe file name and extension, for example foo.2.html, while the mostrecent version does not carry a version number (e.g., foo.html). Serverssubscribing to a changed resource are notified that a new version isavailable.

Dynamic Resource Replication

Since resources are assembled into higher order resources simply byreference, in principle it would be sufficient to retrieve them from therespective Home Servers of the authors. However, there are severalproblems with this simple approach. Since the resource assemblymechanism is designed to facilitate content assembly from a large numberof widely distributed sources, individual sessions would depend on alarge number of machines and network connections to be available, andthus be rather fragile. Also, frequently accessed resources couldpotentially drive individual machines in the network into overloadsituations.

Finally, since most resources depend on content handlers on the AccessServers to be served to a client within the session context, the rawsource would first have to be transferred across the Network from therespective Library Server to the Access Server, processed there, andthen transferred on to the client.

To enable resource assembly in a reliable and scalable way, a dynamicresource replication scheme was developed. FIG. 3 shows the details ofthis mechanism. Anytime a resource out of the resource space isrequested, a handler routine is called which in turn calls thereplication routine (FIG. 3A). As a first step, this routine determineswhether or not the resource is currently in replication transfer (FIG.3A, Step D1 a). During replication transfer, the incoming data is storedin a temporary file, and Step D1 a checks for the presence of that file.If transfer of a resource is actively going on, the controlling handlerreceives an error message, waits for a few seconds, and then calls thereplication routine again. If the resource is still in transfer, theclient will receive the message “Service currently not available.”.

In the next step (FIG. 3A, Step D1 b), the replication routine checks ifthe URL is locally present. If it is, the replication routine returns“OK” to the controlling handler, which in turn passes the request on tothe next handler in the chain.

If the resource is not locally present, the Home Server of the resourceauthor (as extracted from the URL) is determined (FIG. 3A, Step D2).This is done by contacting all library servers in the author's domain(as determined from the Lookup Table, see FIG. 2B). In Step D2 b, aquery is sent to the remote server whether or not it is the Home Serverof the author (in our current implementation, an additional cache isused to store already identified Home Servers (not shown in thefigure)). In Step D2 c, the remote server answers the query with “True”or “False”. If the Home Server was found, the routine continues,otherwise it contacts the next server (Step D2 a). If no server could befound, a “File not Found” error message is issued. In our currentimplementation, in this step the Home Server is also written into acache for faster access if resources by the same author are needed again(not shown in the figure).

In Step D3 a, the routine sends a subscribe command for the URL to theHome Server of the author. The Home Server first determines if theresource is present, and if the access privileges allow it to be copiedto the requesting server (FIG. 3A, Step D3 b). If this is true, therequesting server is added to the list of subscribed servers for thatresource (Step D3 c). The Home Server will reply with either “OK” or anerror message, which is determined in Step D4. If the remote resourcewas not present, the error message “File not Found” will be passed on tothe client. If the access was not allowed, the error message “AccessDenied” is passed on. If the operation succeeded, the requesting serversends an HTTP request for the resource out of the /raw server contentresource area of the Home Server.

The Home Server will then check if the requesting server is part of thenetwork, and if it is subscribed to the resource (Step D5 b). If it is,it will send the resource via HTTP to the requesting server without anyprocessing by content handlers (Step D5 c). The requesting server willstore the incoming data in a temporary data file (Step D5 a); the samefile checked in Step D1. If the transfer is not completed, andappropriate error message is sent to the client (Step D6). Otherwise,the transferred temporary file is renamed as the actual resource, andthe replication routine returns “OK” to the controlling handler (StepD7).

FIG. 3B depicts the process of modifying a resource. When an authorpublishes a new version of a resource, the Home Server will contactevery server currently subscribed to the resource (FIG. 3B, Step U1), asdetermined from the list of subscribed servers for the resourcegenerated in FIG. 3A, Step D3 c. The subscribing servers will receiveand acknowledge the update message (Step U1 c). The update mechanismfinishes when the last subscribed server has been contacted (messages tounreachable servers are buffered). Each subscribing server will check ifthe resource in question had been accessed recently, that is, within aconfigurable amount of time (Step U2).

If the resource had not been accessed recently, the local copy of theresource is deleted (Step U3 a) and an unsubscribe command is sent tothe Home Server (Step U3 b). The Home Server will check if the serverhad indeed originally subscribed to the resource (Step U3 c) and thendelete the server from the list of subscribed servers for the resource(Step U3 d). If the resource had been accessed recently, the modifiedresource will be copied over using the same mechanism as in Step D5 athrough D7 of FIG. 3A (FIG. 3B, Steps U4 a through U6).

Construction of a Course by the Instructor

Content Re-Usage and Granularity

Any faculty participating in the Network can publish their own learningresources into the common pool. To that end, the Network provides a“construction space” which is only accessible to the author, and apublication process, which transfers the material to the shared pool.During the publication process, metadata about the resource is gathered,and system-wide update notification and versioning mechanisms aretriggered.

Learning resources can be simple paragraphs of text, movies, applets,individualizing homework problems, etc. In addition to providing adistributed digital library with mechanisms to store and catalog theseresources, the Network enables faculty to combine and sequence theseresources at several levels. An instructor from Community College Acould combine a text paragraph from University B with a movie fromCollege C and an online homework problem from Publisher D, to form onepage. Another instructor from High School E can take that page fromCommunity College A and combine it with other pages into a module, unitor chapter. Those in turn can be combined into whole coursepacks.Faculty can design their own curricula from existing and newly createdresources instead of having to buy a complete off-the-shelf product.

FIG. 4 shows a general overview of the resource assembly mechanism andthe different levels of content granularity supported by the currentimplementation of this principle. The topmost puzzle piece represents aresource at the fragment level—one image, one movie, one paragraph oftext, one problem, or one regular web page. Attached to the resource ismetadata gathered at the publication time of the resource.

Using the resource assembly tool described below, these fragments andpages can be assembled into a page. A page is a resource of the grainsize which would be rendered as one page on the web and/or on theprinter.

Using the same tool, fragments (which would then be rendered asstand-alone pages), pages, and sequences can be assembled intosequences. Sequences are resources which are rendered a sequence ofpages, not necessarily linear. Examples are one lesson, one chapter, orone learning cycle.

On the third granularity level, fragments (rendered as stand-alonepages), pages, and sequences can be assembled into courses. Courses area sequence which represents the entirety of the resources belonging to alearning unit into which learners can be enrolled. Examples are aUniversity one-semester course, a workshop, or a High School class.

Maps

To increase the utility of the materials, the number of hard-codedhyperlinks between the resources should be minimized. The actualcombining and sequencing is part of the system functionality and drivenby RAT-constructed “roadmaps”, which are constructed by the instructors.With this mechanism, one and the same resource can be part of differentcourses in different contexts. The soft-linking makes it possible toimport only the desired set of resources without effectively importingadditional parts another instructors resources through hard-linked menusor “next page” buttons that might resided on those resources.

Curriculum Adaptivity

Maps allow for conditional choices and branching points. The actual paththrough and presentation of the learning resources is determined byinstructor-specified combinations of learner choices and systemgenerated adaptations (for example, if the learner does not pass a test,additional resources may be included). Each learner can have anindividualized curriculum according to preferences, capabilities andskills.

These maps can be generated at different levels of granularity with agraphical tool, or in an automated way through custom scripts.

Resource Assembly Tool

The Network provides the Resource Assembly Tool as one means to generatemaps. The Resource Assembly Tool provides a graphical user interfaceinside of a standard web browser. The current implementation is writtenin JavaScript™. FIG. 5 shows screenshots of the current implementation.The interface usually consists of two browser windows, one resizable onewith a frameset that contains the menu and the map under construction,and a multipurpose non-resizable window that displays information andinput forms. When a new map is started, it only has a start and a finishresources. The author can then enlarge the map area and insert resourcesinto it.

In FIG. 5A, the author is editing information about a resource in themap after clicking on the box representing the resource in the map. Inthe dialog, the author can enter a map-internal title for the resource,which is displayed to the learners when navigating the maps. In the samedialog, the author will specify the URL of the resource, which caneither be internal to the Network, or any URL of a web page outside ofit. For internal resources, the author can also browse the Networkfilesystem or search the resource metadata to locate an appropriateresource.

The resource priority can be chosen. A resource can be “regular.”“mandatory” or “optional.” These resource priorities are only used inthe bookkeeping of earned points by the learners. Within the map,resources of different priorities are displayed in different colors. Thedialog also allows for two modes of removing the resource from the map:either deleting it from the map including every link to and from it, ordeleting it while reconnecting any links that went through the resource.As an example, resources A and B might both connect to resource C, andresource C might connect to D. When removing C from the map using thefirst option, both A and B will not no longer be connected to D. Usingthe second option, both A and B will reconnect with D. In the lattercase, the Resource Assembly Tool will also handle conditional linkscorrectly: such as, if A connected to C under condition 1, and Cconnected to D under condition 2, then in the end A will connect to Dunder a new condition which is (1 AND 2).

Finally, this dialog allows the author to connect the resource toanother resource (or itself) through a new link. When selecting thisoption, the Resource Assembly Tool goes into link mode, and will linkthe current resource to the next clicked resource (unless the action iscancelled).

FIG. 5B shows the Resource Assembly Tool in info mode, that is, when nospecific component of the map is edited, and if the Tool is not in linkmode. In info mode, the contents of the dialog window change dynamicallyas the mouse is moved over the components of the map. In this case, themouse pointer is over the link condition between two resources. Thedialog window shows the titles of the connected resources, as well asthe condition priority. In this scenario, the condition priority is setsuch that the link cannot be taken (i.e., “is blocked”) if the conditionis false. The condition priority can also be set such that the link isrecommended if the condition is true (possibly giving the learnerseveral options where to go next), or that the link must be taken (“isforced”) over any other possible link if the condition is true. Withinthe map, conditions of different priorities are displayed in differentcolors. If the author now were to click on the condition, the Tool wouldgo into edit mode, and the condition could be edited.

FIG. 5C shows the Tool in edit mode for the link between the resourcetitles displayed. The author can remove the link, or insert a newresource into the link. Obviously, by this mechanism, rather complexmaps can be generated. These are different from binary trees, bothbecause branches can loop back, and because branches can be re-united.In fact, most branches re-unite in the finish resources. Into each link,a condition with one of three different priorities can be attached.Whether or not a certain resource in the map can be displayed depends onwhether or not it can be reached through any path along allowed links,starting with the start resource of the course. If a resource is notlinked to, it is assumed to be accessible if the map which it is part ofis accessible.

Map Representation and Storage Format

FIG. 6 shows the XML representation of the resource map constructed inFIG. 5, which is the format in which maps are stored. In the figure,however, additional graphical map layout information generated by theResource Assembly Tool is not displayed. This graphical information isoptional to re-generate the same graphical layout when the map isbrought up again in the Resource Assembly Tool, and is not needed forany other system functionality.

Maps can be generated by tools other than the Resource Assembly Tool. Inparticular, an author might have some other representation of a coursesequence, which can be converted into a map using scripts. If this mapthen were to be brought up in the Resource Assembly Tool, the Tool wouldautomatically generate a graphical layout for it. Each entry of the map,resources, conditions and links, are stored in separate tags.

Resources and conditions have to have unique ID numbers. These numbersare automatically generated by the Resource Assembly Tool when the entryis first created, or added to the entries when a map generated outsidethe Resource Assembly Tool is first retrieved. They can also be assignedby custom scripts or added in by hand.

In this example, FIG. 6, entry 1 is the start resource of the map. Whenthis map is accessed, the source (src) URL of this tag will be the firstresource rendered. Entry 2 is the finish resource of this map. Thisresource will be the last resource in the sequence of resources. Entry 6is a problem resource with the given URL and title, as well as thepriority “mandatory”. Entry 19 is a condition, which is used by the linkbetween entries 6, the problem, and 9, a sequence.

Example of Nested Maps

FIG. 7 shows the XML representation of three maps which are importedinto each other. FIG. 7B is the sequence that is referenced as resource9 in the course map FIG. 7A. In the resulting map, the entry point ofresource 9 in FIG. 7A is in fact the entry point of the start resourceof FIG. 7B, namely, resource 1. The exit point of resource 9 in FIG. 7Ais the exit point of the finish resource of FIG. 7B, namely, resource 2.FIG. 7C is the page which is referenced as resource 24 in FIG. 7B.

A course can easily contain several hundreds of these nested maps. Sincethe accessibility of each individual resource in the course depends onthe state of all possible paths linking it to the start resource of thecourse across all intermediate maps, the computation and disk-I/O effortper single transaction could quickly become prohibitive. Thus, all mapsand conditions are compiled into a pre-processed binary data structureat the start of a session.

Initialization of a Course for a Learner

When a learner first enters a course during a session, the system willinitialize this course for the learner. In particular, at this point,the course map and all nested (embedded) maps and resources areevaluated, and the information is compiled into two binary structures,which are stored with the session information: the resource propertieshash, and the link conditions array. This information will be used overthe duration of the session for several purposes: navigation (whichresource is the next, which one the previous?), for access control (canthe resource be reached under the link conditions given the currentstate of the student?), and to register assessment results within thecontext of a certain course and map (there might be several instances ofthe same problem resource within a course).

Evaluation of the Map Structure for a Course

The URL of the course is passed to the procedure readmap (FIG. 8).Procedure readmap first initializes the resource properties as an emptyhash, seeds the link conditions array with a 0th element, which is setto “true”, priority “normal”, and sets the map counter to 0 (FIG. 8,Step R1). While the resource properties hash, the link conditions arrayand the map counter are global variable of the initialization process,all other variables are local to the procedures (an important propertyfor these routines to run recursively). The procedure readmap then callsprocedure loadmap for the URL of the course (FIG. 8. Step R2).

FIGS. 9 & 10 show a dump of excerpts of the binary structure generatedin loadmap for the nested maps of FIG. 7. Procedure loadmap (FIG. 9A)first checks if the map URL has already been processed (multipleinclusion of the same map in a course structure) (FIG. 9A, Step L1). Ifthe URL was processed, it has been assigned a map counter value in theresource properties hash. If the map has been processed, there is noneed to process it again, and loadmap returns.

If the map has not been processed, the map counter is incremented andthe map is registered under the current value in the resource propertieshash (FIG. 9A, Step L2). The file is then opened (FIG. 9A, Step L3),which might entail prior replication, and the contents are parsed. Ifthere are no further entries. loadmap returns (FIG. 9A. Step L4).

The new entry tag is then read (FIG. 9A. Step L5) and the type isdetermined (FIG. 9A, Step L6). If the entry is a resource (Step L7), aresource ID is formed by combining the map counter and the resource IDwithin the map. For example, the “Part 1 Introduction” resource ofpart1.sequence (FIG. 7B) was assigned the resource ID 2.5, since it hasthe internal resource ID 5 in the 2^(nd) map processed (see FIG. 9Bunder “ids_”). If the same URL is found again, additional IDs areassigned to it. It is necessary to store the IDs under the URL in theresource properties hash for reverse lookup if a user simply requests aURL. If the resource is a start or finish resource, the resource ID isregistered as the start or finish resource of the map, respectively(FIG. 9B. “map_start”. “map_finish”). The properties of the resource(URL, Title, Priority, etc) are now stored under the resource ID (seefor example FIG. 9B “title_(—)2.5”).

If the resource is not a map itself (FIG. 9A, Step L8), the next entryis read. Otherwise, procedure loadmap calls itself recursively toprocess that map (Step L9). If in Step L6, the type of the entry wasdetermined to be a condition, a condition ID is formed (Step L10) byagain combining the map counter with the internal ID. The condition isalso added to the end of the condition array (see FIG. 10), which is acompilation of all conditions in the course (Step L11). The conditionsin this array are evaluated when a transaction occurs that could changethe state of the student, and the state of each condition is stored bythe index number in the session environment. A reference to the indexnumber in the condition array is stored under the condition ID (FIG. 9D.“condid_”).

If the entry is a link (Step L6), a link ID is generated (Step L12).This ID is formed by combining the map counter and another counter whichis incremented for every new link within the map. Under this ID, the IDsof the originating and the destination resource of the link are stored,as well as that of the link condition (FIG. 9D). For the originatingresource, in Step L13 the link ID is added to the list of outgoing links(FIG. 9C, “to_”), and for the destination resource, the link ID is addedto the list of incoming links (FIG. 9C, “from_”).

After the last entry has been processed, procedure loadmap returns.After the last map has been processed, the original course-levelinstance of loadmap returns to readmap (FIG. 8, Step R2). The next majorstep will be to determine all possible paths and conditions leading upto a resource for access control.

readmap checks if the course has a start resource from its map_startentry in the resource properties (Step R2), and if does not, continue tostore the two global binary data structures (Steps R5,R6). In thisspecial case, all resources which are part of any maps in the course areassumed to be accessible.

If the course has a start resource, readmap calls the proceduretraceroute (FIG. 11A) with the following parameters (Step R4); 1) thecumulative condition along this path or route so far is set to “true”(the map is accessible), 2) the resource ID of the start resource of thecourse map, and 3) an empty list for all resources processed so faralong this route. It is again important to note that all variablesexcept the global binary structures are local to traceroute, sincetraceroute will recursively call itself whenever there is a branching tofollow all possible paths of the maps.

traceroute will establish a section within the resource properties hashthat builds up all conditions leading up to a resource. FIG. 11 B showsan excerpt of the final result. For example, resource 2.5, theintroduction to part 1, can be reached under condition 8 (see FIG. 10),meaning, after solving the pretest problem. traceroute first checks ifthe resource has already been processed on this route by its resource ID(FIG. 11A. Step T1). This test avoids that traceroute runs into endlessloops when the links on the map loop. Next, the resource ID is added tothe list of processed resources on this route (Step T2).

The resource conditions are now OR'd (i.e. a logical “or” is appled)with the cumulative conditions on this route (Step T3)—the routerepresents another way of getting to the resource. A small routine withsimplification rules for boolean expressions is called to simplify thepotentially very long expression.

In the next step, it is determined if the resource is itself a map (StepT4). If it is, the exit route conditions can differ from the entry routecondition by all additional conditions along the paths in the embeddedmaps (for non-map resources, entry and exit route conditions are thesame). If, however, the embedded map does not have a start resource(Step T5), that is not the case—again, the missing of entry point to anembedded link structure is interpreted as the resources being openlyaccessible.

If the embedded map does have a start resource. traceroute is calledrecursively with the current route conditions, the ID of the startresource of that map, and the list of already processed resource IDs(Step T6). Upon return, if the embedded map does not have a finishresource, the entry and exit conditions of this map are assumed to bethe same (Step T7). If the map had a finish resource, the routecondition so far is set to the resource condition of the finish resourceof the embedded map (Step T8). In order go on from here, the user wouldhave had to reach the finish resource of the embedded map.

Now the route conditions are correctly set for exiting the resource andgoing on from here. traceroute now loops over all outgoing links of theresource (Step T9). If the link does have a link condition (Step T10),then the route condition for this branch path is the cumulative routecondition so far AND the link condition (Step T1 I). If there is no linkcondition, then there is no change in route conditions (Step T12). Tofurther process the routes along this link, traceroute is calledrecursively with the resource ID of the destination resource of thelink, the new route conditions, and the list of already processedresources (Step T13). traceroute returns after processing the lastoutgoing link of the resource it had been called.

FIG. 11B shows part of the output of traceroute for the example FIG. 7.

Multivalued Boolean Evaluation of Link Priorities

When a user accessed a resource on a map and desires to access the“next” resource, the request is processed by a number of steps. From thedata exemplified in FIG. 9C, it is determined which outgoing linksexist. From the data in FIG. 9D it is determined to which resourcesthose links lead. For each of the resources, the expressions in FIG. 11Aare evaluated as follows. Stored in the session environment is theevaluation of the table FIG. 10, where the boolean part is evaluated as“0” or “1”. In addition, a multivalued boolean value is computedincorporating the condition priority. A false blocking condition isassigned the value zero, all other false conditions the value 1. A trueforced condition is assigned the value 3, all other true conditions thevalue 2.

In the expressions FIG. 11A an “&” (“AND”) is processed as the minimum(min) operation, a “I” (“OR”) is processed as the maximum (max)operation. The outcome “0” means “blocked”, the outcome “1” means “notrecommended”, the outcome “2” means recommended, and the outcome “3”means forced.

From the above, it should be clear that the methods and systems of thepresent invention provide computer-driven system that adapts to newinformation, allows for new information to be readily utilized byinstructors, so that the instructor can integrate new information intothe existing curriculum.

We claim:
 1. A computer-implemented apparatus to facilitate creation ofeducational information comprising: one or more processors; one or morememories connected to said one or more processors; the one or morememories storing instructions that, when executed , cause the one ormore processors to perform the steps of: storing at least one resourceand a first electronic file storing metadata information about usage ofthe at least one resource; displaying resource selection information toan educational information creating instructor about said at least oneresource; wherein said resource selection information includes saidmetadata information; organizing and storing identity of resourcesincluded for dissemination to learners, by storing the educationinformation in a resource map data structure; receiving at least one ofsaid learners usage information of the at least one resource in aclient-server session coincident with said learner's use of the resourceafter the resource has been disseminated; determining a degree ofdifficulty of the at least one resource based on received at least oneof said learners usage information; receiving at least one of saidinstructors usage information of the at least one resource prior todissemination of the resource to learners; determining an amount ofusage of the at least one resource by other instructors based on thereceived at least one of said instructors usage information; generatingaugmented resource selection information based on determined degree ofdifficulty of at least one resource and determined amount of usage ofthe at least one resource by other instructors; and displaying saidaugmented resource selection information to said educational informationcreating instructor.
 2. The apparatus of claim 1 further comprisingstoring metadata information in the first electronic file about usage ofthe at least one resource by the educational information-creatinginstructor.
 3. The apparatus of claim 1 further comprising storingmetadata information in the first electronic file about usage of the atleast one resource by the learner.
 4. The apparatus of claim 1 whereinthe augmented resource selection information catalogs the degree ofdifficulty of the resource.
 5. The apparatus of claim 1 furthercomprising determining the degree of difficulty based on a degree towhich the resource discriminates among learners based on the learners'abilities.
 6. The apparatus of claim 1 further comprising determiningthe degree of difficulty to assess how well the resource reflectsoverall ability of the learner.
 7. The apparatus of claim 1 wherein theaugmented resource selection information catalogs the degree ofdifficulty and discrimination for learners of different ability levelsof the at least one resource based on captured information from at leastone of said learners.
 8. The apparatus of claim 1 wherein the augmentedresource selection information lists usage by instructors in linking anoriginating resource with a destination resource.
 9. The apparatus ofclaim 1 further comprising storing said educational information bynesting individual resources to form another resource.
 10. The apparatusof claim 1 further comprising: storing said educational information bynesting individual resources to form another resource and storing saidanother resource in association with a second electronic file thatstores metadata information about usage of said another resource. 11.The apparatus of claim 1 wherein the metadata information stored in saidfirst electronic file keeps track of the learner's participation andperformance.
 12. The apparatus of claim 10 wherein the metadatainformation stored in said second electronic file catalogs the degree ofdifficulty of the another resource.
 13. The apparatus of claim 10wherein the metadata information stored in said second electronic filecatalogs the degree of difficulty of the another resource based oncaptured information from at least one of said learners.
 14. Theapparatus of claim 10 wherein the metadata information stored in saidsecond electronic file reflects the degree to which the another resourcediscriminates among learners based on the learners' abilities.
 15. Theapparatus of claim 10 wherein the metadata information stored in saidsecond electronic file discriminates how well the another resourcereflects overall ability of the learner.
 16. The apparatus of claim 10wherein the metadata information stored in said second electronic filecatalogs the degree of difficulty and discrimination for learners ofdifferent ability levels of the another resource based on capturedinformation from at least one of said learners.
 17. The apparatus ofclaim 10 wherein the metadata information stored in said secondelectronic file catalogs the amount of usage of the another resource byother instructors in the past.
 18. The apparatus of claim 10 wherein themetadata information stored in said second electronic file lists usageby instructors in linking an originating resource with a destinationresource.
 19. The apparatus of claim 10 wherein the metadata informationstored in at least one of said first and second electronic files keepstrack of the learner's participation and performance.
 20. Acomputer-implemented method to facilitate creation of educationalinformation, comprising storing at least one resource and a firstelectronic file storing metadata information about usage of the at leastone resource; displaying resource selection information to aneducational information creating instructor about said at least oneresource; wherein said resource selection information includes saidmetadata information; organizing and storing by a computer identity ofresources included for dissemination to learners, by storing theeducation information in a resource map data structure; receiving atleast one of said learners usage information of the at least oneresource in a client-server session coincident with said learner's useof the resource after the resource has been disseminated; determining bythe computer a degree of difficulty of the at least one resource basedon received at least one of said learners usage information; receivingby the computer at least one of said instructors usage information ofthe at least one resource prior to dissemination of the resource tolearners; determining by the computer an amount of usage of the at leastone resource by other instructors based on the received at least one ofsaid instructors usage information; generating by the computer augmentedresource selection information based on determined degree of difficultyof at least one resource and determined amount of usage of the at leastone resource by other instructors; and displaying by the computer saidaugmented resource selection information to said educational informationcreating instructor.
 21. A non-transitory computer-readable mediumstoring instructions that, when executed, cause one or more processorsto perform the steps of: storing at least one resource and a firstelectronic file storing metadata information about usage of the at leastone resource; displaying resource selection information to aneducational information creating instructor about said at least oneresource; wherein said resource selection information includes saidmetadata information; organizing and storing identity of resourcesincluded for dissemination to learners, by storing the educationinformation in a resource map data structure; receiving at least one ofsaid learners usage information of the at least one resource in aclient-server session coincident with said learner's use of the resourceafter the resource has been disseminated; determining a degree ofdifficulty of the at least one resource based on received at least oneof said learners usage information; receiving at least one of saidinstructors usage information of the at least one resource prior todissemination of the resource to learners; determining an amount ofusage of the at least one resource by other instructors based on thereceived at least one of said instructors usage information; generatingaugmented resource selection information based on determined degree ofdifficulty of at least one resource and determined amount of usage ofthe at least one resource by other instructors; and displaying saidaugmented resource selection information to said educational informationcreating instructor.